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Abstract — In this paper, we introduce a sparse approximation 
property of order s for a measurement matrix A: 



<D||Ax||2 + /3 



(Ti,(x) 



for all X, 



where Xs is the best s-sparse approximation of the vector x in ^^, 
as(x) is the s-sparse approximation error of the vector x in i?^, 
and D and /3 are positive constants. The sparse approximation 
property for a measurement matrix can be thought of as a 
weaker version of its restricted isometry property and a stronger 
version of its null space property. In this paper, we show that 
the sparse approximation property is an appropriate condition 
on a measurement matrix to consider stable recovery of any 
compressible signal from its noisy measurements. In particular, 
we show that any compressible signal can be stably recovered 
from its noisy measurements via solving an £^ -minimization 
problem if the measurement matrix has the sparse approximation 
property with f] G (0, 1), and conversely the measurement matrix 
has the sparse approximation property with /? e (0, oo) if 
any compressible signal can be stably recovered from its noisy 
measurements via solving an £^ -minimization problem. 



I. Introduction 

Given positive integers m and n with m < n and a 
measurement matrix A of size mxn,we consider the problem 
of compressive sampling in recovering a compressible signal 
X £ M" from its noisy measurements z = Ax + n via solving 
the following ^^-minimization problem: 



min||y|j^ subject to ||Ay 



< e, 



(I.l) 



where < q < l,q < p < oo,e > 0, and the measurement 



noise n satisfies ||n||p < e (11) - |8|). Here 



},0 < q < oo. 



stand for the "f^-norm" on the Euclidean space. 

Given a subset S C {!,..., n} and a vector x e R", 
denoted by xs the vector whose components on S are the same 
as those of the vector x and vanish on the complement S*"^. A 
vector X G R" is said to be s-sparse if x = xg for some subset 
S C {1, . . . ,n} with its cardinality #5* less than or equal to 
s, where s > 1. Denote by E^ the set of all s-sparse vectors. 
Given a vector x, its best s-sparse approximation vector x^ in 
£'^ is an s-sparse vector which has minimal distance to x in 
£«;i.e., Hx-x^Hq = crs,g(x) := infyes, Hx-yH^. For g == 1, 
we use crs(x) instead of crs^i(x) for brevity. 
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In this paper, we introduce a new property of a measurement 
matrix A: there exist positive constants D and /3 such that 

||x,||^ < D\\Ax\\l + /3s«/'-i(a,,,(x))« for all x e M", 

(1.2) 
where < p,q,r < cx), s is a positive integer, and x^ is the 
best s-sparse approximation of the vector x in £''. The property 
of a measurement matrix mentioned in the abstract is a special 
case of the above property where p = r — 2 and q = I. We call 
the property ( II.2b the sparse approximation property of order 
s, as it is closely related to the best s-sparse approximation. 
We call the minimal constant (3 such that (II. 2b holds the sparse 
approximation constant, and denote it by /3s (A). 

In this paper, we show that for the stable recovery of a com- 
pressible signal X from its noisy measurements z = Ax + n 
via solving the ^^'-minimization problem ( II. lb . the sparse ap- 
proximation property ( II. 2b with sparse approximation constant 
/?s(A) < 1 is sufficient while the sparse approximation 
property ( II.2b with finite sparse approximation constant /3s (A) 
is necessary. We refer the reader to Il2l. ||3l. llTl. Il9l - IfTTl 
and the references therein for other various conditions on a 
measurement matrix that guarantee the stable recovery of any 
compressible signal from its noisy measurements via solving 
the £'' -minimization problem ( 11.11 ). 

Theorem 1.1: Let 0<g< l,<?<r< oo, 1 < p < oo, 
e > 0, positive integers m, n, s satisfy 2s < m < n, A be a 
matrix of size mxn having the sparse approximation property 
(ini i with D e (0, oo) and {3 e (0, 1), z = Ax + n with 
\\n\\p < e and x £ R", and let x* be the solution of the 
^'-minimization problem (II. lb . Then 



and 



? < 



l?< 



(3 + /3)£> 
1-/3 



(2e)' 



2(l + /3)2 



1-/3 



(1.3) 



(3 + /?)^ .i-,/p 
1-/3 



(26) 



2(1 + /?)^ 



I-P 



{<r. 



(1.4) 



if g < r, and 



xll' < - 



2D 



{2ey 



2(1 + /?) 



(a,,,(x))« (1.5) 



/3^ ' 1-/3 

if q — r. 

Theorem 1.2: Let < g,^ < oo, positive integers m,n,s 
satisfy 2s < m < n, and let A be a matrix of size mxn. 
If for any e > and x e R", the error between the given 
vector X and the solution x* of the ^'-minimization problem 
(II. lb satisfies 

llx* - x|l^ < Bie' + i32s'/P-i(a,,,(x))', (1.6) 



where Bi and B2 are positive constants independent of e and 
X, then 

llxll? < Bi||Ax||^ + B2si^P-\as,g{^)y for all x G R", 

(1.7) 
and hence A has the sparse approximation property il.2\ with 
r =p, D^ Bi and 13 = B2. 

The rn X n adjacency matrix $ of an unbalanced (2s, a)- 
expander with left degree d and a G (0, 1/4) satisfies 

1 2a 

||x,||i < — -— ||*x||i + Y3^^s(x) for all x e R", 

(1.8) 
(and hence it has the sparse approximation property (11.21 ) with 
p = q = r = 1). The above property for the adjacency matrix 
$ is established in lfT2l Lemma 16] implicitly. Then by (II.8I 1 
and Theorem ll.il we have the following result similar to lfT2l 
Theorem 17]. 

Corollary 1.3: Let e > 0, positive integers m, n, s satisfy 
2s < m < n, a e (0, 1/6), $ be the m x n adjacency 
matrix of an unbalanced (2s, a)-expander with left degree d, 
z = #x + n with ||n||i < e for some x G M", and let x* be 
the solution of the minimization problem (II. lb with p = q = 1. 
Then 

ll-*--ll^^d(T^^ + B^-W- ^'-'^ 

The paper is organized as follows. One of two basic proper- 
ties of a measurement matrix A in compressive sampling ([18] 
- 02411 ') is the null space property of order s in i'' ,0 < q < 1; 
i.e., there exists a positive constant 7 such that 



iP to £1, i.e.. 



IRxIl, < 



IRI 



p-i-gl 



x||p for all vectors x. In this 



|xs||^ < 7l|x5<= 



(1. 10) 



hold for all vectors x in the null space N{A) of the matrix 
A and all sets S with cardinality #5* less than or equal to s. 
In Section nil we show in Theorem 12. II that any measurement 
matrix satisfying ( 11.21 ) will have the null space property dl.lOl ). 
So the sparse approximation property (11.21 ) of a measurement 
matrix can be considered as a stronger version of the null 
space property ( 11.101 ). The other basic property of a measure- 
ment matrix A in compressive sampling (III, 121, Q, fTS] - 
11241 ) is the restricted isometry property of order s; i.e., there 
exists a positive constant 6 £ (0, 1) such that 

(1 - 6)\\x\\l < |jAx||2 < (1 + d)\\x\\l for all x e E.. 

(1. 11) 
In Section [nil we prove that if a measurement matrix has 
the restricted isometry property ( II. lib of order 2s then it has 
the sparse approximation property (11.21) with p = r = 2, 
and furthermore the constant /? in ( 11.21 ) is small when the 
restricted isometry constant is small, see Theorems l3.1l and l3.2l 
for details. Thus the sparse approximation property ( II. 2b of a 
measurement matrix can also thought of as a weaker version 
of the restricted isometry property ( II. lib , see also Remarks 
13.31 and 13.41 The proofs of all theorems are included in the 
appendix. 

II. Null space property and sparse approximation 

PROPERTY 

Let 7?,(A) be the set of matrices R satisfying A = ARA, 
and denote by ||R||p^q the operator norm of a matrix R from 



section, we show that any measurement matrix satisfying ( 11.21 ) 
will have the null space property ( ILlOb with its null space 
constant less than or equal to the constant /? in ( II. 2b . Here 
null space constant 7s (A) of a measurement matrix A is the 
minimal constant 7 such that ( ILlOb holds. 

Theorem 2.1: Let < g < r < 00, Q < p < 00, integers 

TO, n, s satisfy 2 < 2s < m < n, and A be a matrix of size 

m X n. Then the following statements hold. 

(i) If the matrix A has the sparse approximation property 

( 11.21 ). then it has the null space property of order s in ^' 

with its null space constant 7s (A) < /3s (A). 

(ii) If the matrix A has the null space property of order s 

in (f^ with the null space constant 7s (A), then it has the 

sparse approximation property ( 11.21) with p = q = r, D ~ 

max(l,7s(A))infR.g7^(A) HRH^^, and (3 = 7s(A); i.e.. 



Ilxsll^ < (max(l,7s(A)) inf 

^ ^ ReK(A) 



|R|IU)l|Ax||? 



+7s(A)(fTs.g(x))'' for all x e R". 

Applying Theorems 11.11 and 12.11 with p = q = r = 1, we 
have the following result on recovering compressible signals 
from noisy measurements when the measurement matrix has 
the null space property of order s in ^^, which is obtained in 
[8] for the noiseless case. 

Corollary 2.2: Let e > 0, m,n,s be positive integers with 
2s < TO < n, A be a matrix of size to, x n satisfying the null 
space property ( II. 10b with q = 1, z = Ax + n with ||n|| 1 < e 
and X e R", and let x* be the solution of the minimization 
problem dl.lb with p = q = 1. If the null space constant 
7s(A)G (0,1), then 



x||i< 



4 inf 



ReK(A) l|R|li^i , 2 



l-7s(A) 



2% (A) 
1-7. (A) 



(Ts{x.)- 



Remark 2.3: The null space property of a measurement 
matrix is invariant under preconditioning, i.e., if a measure- 
ment matrix A has the null space property dl.lOl ) then the 
preconditioned matrix PA has the null space property ( 11.101 ) 
with the same null space constants, where a preconditioner is 
a nonsingular matrix P. The sparse approximation property 
( |I.2b is weakly preconditioning-invariant in the sense that if 
a measurement matrix A satisfies (II. 2b then the precondi- 
tioned matrix PA also satisfies ( 11.21 ) with D replaced by 
||P^^||p^pD. This suggests appropriate preconditioning the 
measurement matrix (and hence the noisy measurements) 
before signal recovery from its noisy measurements via solving 
an ^''-minimization problem. 

Remark 2.4: Let the matrix A of size mxn have full rank 
TO (which is the case in most of compressive sampling prob- 
lems) and A = UEV* be its singular value decomposition. 
Here and hereafter x* stands for the transpose of a vector or 
a matrix x. Then E = (E' 0) for some nonsingular diagonal 
matrix E'. Now we define the conventional preconditioned 
measurement matrix A by A = PA, where P = (E')^^U*. 

In this case, R e 7?. (A) if and only if R = V( _ ], where 

I is the unit matrix of size m x to, and B is an arbitrary 
matrix of size (to, — n) x n. Let v^ , 1 < i < n, be the column 



vectors of the matrix V. Then the null space N{A) of the 
matrix A is spanned hy Vi,m + 1 < i < n, and the vectors 
Vi,! < i < m, form an orthonormal basis for A^(A)-'-, the 
orthogonal complement of the null space N{A) of the matrix 
A. As the set {Rx : ||x||i<l}isa polygon, the maximal 
£^-norm of Rx, ||x||i < 1, is then attained at some vertices. 
Thus 



inf ||R||i^.i = inf max | 

ReK(A) ReK(A) l<i<rn 

Vi - (v,„+i • • • v„)Bej 



Re; 



inf max 

B l<i<m 



inf max 

ueAf(A) l<i<m 



|v., -u||i, 



(II. 1) 



where e^, 1 < i < m, form the standard orthonormal basis of 
R™. In other words, the quantity infj^g^/^N ||R||i^i is the 
same as the distance of v^, 1 < i < tti, from the null space 



IRII 



< 



iV(A) in £^. From (III. Il l it follows that infj^g.;^,^) 
maxi<i<m II Villi £ n^/^. It would be an interesting topic on 
preconditioning a measurement matrix A with the null space 



ReTC(A) 



R-llg^g: < 



property ( II. 10b such that the quantity inf 

q < 1, for the preconditioned matrix A is not a large number. 

III. Restricted isometry property and sparse 

APPROXIMATION PROPERTY 

In this section, we prove that if a measurement matrix has 
the restricted isometry property dl.lll i of order 2s, then it has 
the sparse approximation property jl.lj with p ^ r — 2, and 
the sparse approximation constant is small when the restricted 
isometry constant is small. Here the restricted isometry con- 
stant Ss (A) of a measurement matrix A is the smallest positive 
constant 6 that satisfies (11.111 ). 

Theorem 3.1: Let < q < 1, positive integers 111,71,3 
satisfy 2s < m < n, and the matrix A of size mx n have the 
restricted isometry property (II. lib of order 2s with restricted 
isometry constant ^25 (A) £ (0, 1). Then for all x e M", 



2 ^ v/l + ,52.(A) + v/2MA) 2 

'' - (i-52.(A))yTTMA) ' 



^l + S2siA) + ^2S2s{A)^^ 

l-S2siA) 

x52s(A)si-2/9(a,.,(x))' 



(III.l) 



and 



|Ax||^ < 



(l + <52s(A) + y2MA))||x|| 
+ {l + ^252s{A))S2s{A) 
xsi-2/9(^^^^(x))2. 



(111.2) 



Theorem 3.2: Let 0<g<r<oo,0<p<oo, positive 
integers m, n, s satisfy 2s < m < n, and A be a matrix of 
size m X n that has the sparse approximation property (II. 2b . 
Then 

(III.3) 



-||x||,« < ||Ax||« forallxeS, 



and 



1^1 
2D ' 



x||«<||Ax||^ forallxeE2. 



(111.4) 



Remark 3.3: From Theorem |3.1| we see that a measurement 
matrix with small restricted isometry constant will have the 
sparse approximation property (II. 2b with p ~ r — 2,D close 
to one and (3 close to zero. Conversely for p = r = 2 we 
obtain from Theorem l3.2l that if a measurement matrix A has 
the sparse approximation property ( II. 2b with D close to one 
and (3 close to zero, then the first inequality in the restricted 
isometry property ( II. lib holds for some constant S close to 
1/2 only. For p = g = r = 1, the m x n adjacency matrices 
# of unbalanced (2s, a)-expander with fixed left degree d 
has the sparse approximation property (II. 2b with small sparse 
approximation constant (see (II. 8b ) and the restricted isometry 
property with respect to £^-norm: 

(1-Ca)||x||i < ||*x||i/d < (1 + Ca)||x||i for all x e S2. 

where C is a positive constant (see lfT2l Theorem 1]), but 
it does not have the restricted isometry property dl.llb when 
to/s^ is sufficiently small f27l|. 

Remark 3.4: If a measurement matrix A has the restricted 
isometry property (II. lib with small restricted isometry con- 
stant (see [1], |4|, |24l, |l25l, ||26l for examples of such 
measurement matrices), then the preconditioned measurement 
matrix PA has the sparse approximation property (II. 2b with 
p — r = 2, D close to ||P~^||2^2 and (3 close to zero but it 
does not have the restricted isometry property ( II. lib in general. 
This observation may suggest that preconditioning procedure 
could generate new measurement matrices for the stable recov- 
ery of compressible signals from their noisy measurements. 

IV. Conclusions and final remarks 

In this paper, we introduce the sparse approximation prop- 
erty ( II. 2b of a measurement matrix and show that it is a suf- 
ficient and almost necessary condition that any compressible 
signal can be stably recovered from its noisy measurements 
via solving the ^'-minimization problem ( II. lb . 

The sparse approximation property ( II. 2b of a measurement 
matrix with g < r is a stronger version of the null space 
property (II. 10b with the preconditioning-invariance almost pre- 
served. The sparse approximation property (II. 2b with p — r ^ 
2 and < q < 1 is a weaker version of the restricted isometry 
property ( II. lib . The adjacency matrices of some unbalanced 
expanders have the sparse approximation property ( II. 2b with 
p = q = r = 1 and small sparse approximation constant, 
but they do not have the restricted isometry property (II. lib . 
A challenging problem is the construction of measurement 
matrices, other than random matrices Q. El, ll24l . ll25l . ll26l 
and adjacency matrices of a graph fTT\, fTJ], f28l, f29l, f30l, 
that have sparse approximation property (II. 2b with small sparse 
approximation constant. 

Appendix 



A. Proof of Theorem 17.71 

Set h = X* — X. Let So be so chosen that ||x5c||q = ||x — 
Xsllg, Si be the set of indices of the s largest components, in 
absolute value, of h in 5n, 6*2 be the set of indices of the next 



s largest components, in absolute value, of h in (5*0 U SiY, 
and so on. Then 

||Ah||p<2e and ||h5e||^ < HhsJI^ + 2||x-x,||^ (A.l) 

by ( IlT] ). and 

||h5,||.<si/^~i/'|lh5,_JU, j>2 (A.2) 

by the construction of the sets Sj,j > 1, where q < f < r. 
Combining ( 11.21 ) and (lA.ll ) gives 



B. Proof of Theorem 17.21 

The conclusion ( 11.71 ) follows from the estimate (11.61 ) and 
the observation that the zero vector is the solution of the i''- 



minimization problem (II. Il l with e 

any x e K". 



for any subset T of {1, . . . , n} with #r < s. Applying ( IA.3b 
with T replaced by Sq and then using the estimate dA.lb for 

llhscll? < D{2eY + 2/3s^/^-i||x- x,||^ + Ps^/^-^hsA^. 

(A.4) 
By Holder inequality and the property that H^Sq < s. 



\hsJ,<s'^^-'/^\\h 



SoWr- 



(A.5) 



Substituting the above inequality into the right-hand side of 
the inequality ( IA.4I ) leads to the first crucial inequality: 



h5oll?< 



D 



(26)" 



2/3 



s9/'-^i||x-x,||«. (A.6) 



Combining ( lA.ll ). ( IA.5I ) and (IA.6I 1 yields the second crucial 
inequality: 



Ihc,||«< 



D 



,1-g/r 



(26) 



(A.7) 



For r = q, the conclusion ( II.5b follows from (IA.6b and (IA.7 
Applying ( IA.3I ) with T replaced by Si yields 



This together with dA.ll ). ( IA.5I ). ( IA.6I ) and ( |A.7b leads to the 
third crucial inequality: 

||hsj|« < i?(26)' + /?s«/--i(||hsj|« + ||h(5,u5,)=ll?) 



< 



i?(l + /3) 



l^(2er.^^.^/-^l|x-x., 



-/3 



(A.8) 



!h||^ < llhsolll- 



is,ll~ 



Therefore for q < r < r, 

Ice 



-s'/'-'llh5sll? 



< 



2(1+^ /,_ 



(A.9) 



where the first inequality holds by the triangle inequality for 
II • ll'^y- as g < 7~, the second inequality is true by Holder 
inequality and ( |A.2b . and the third inequality follows from 
( IA.6I ). ( |A.7b and (|A.8b . Then the conclusions (|L3]i and ( |L4l i 
follow by letting f = r and f = g in ( IA.9I ) respectively. 



|Ax|L and z = Ax for 



C. Proof of Theorem \2.1\ 

(i) Take a vector x € M" with Ax = and let x^ 
be its best s-sparse approximation in P. Then it is a best 
s-sparse approximation in F . This together with the sparse 
approximation property ( II. 2b leads to ||xs||^ < s^^^/'"||xs||^ < 
/3s(A)(CTs.g(x))'^. Thus the measurement matrix A has the null 
space property of order s with 7s (A) < /3s (A). 

(ii) Take a vector x e K". Then it suffices to prove that 



xtII 



< 



max(l,7s(A))^jnf^J|R||^^J|iAx||^ 



RG7?,(A) 



+7s(A)||xTe||',, 



(A. 10) 



for all subsets T C {1, . . . , n} with #r < s, where < p < 
cx). Note that A(x - RAx) = (A - ARA)x = for all 
R e 7?.(A). This together with the null space property ( II. 10b 
of the measurement matrix A leads to ||(x — RAx)t||? < 
7s(A)|j(x - RAx)THIg for all subsets T C {1, . . . ,n} with 
#T < s and R e 7^(A). Hence 



|xT||^<max(l,7s(A))||R||^^,||Ax||^ 



7.(A)||xTcll' 



Taking minimum over all matrices R G 7?. (A) in the right- 
hand side of the above estimate leads to (lA.lOb . and hence 
proves the second conclusion. 



D. Proof of Theorem 13.71 

Take a vector x e K" and let Xs be its s-sparse approxi- 
mation in i"^. We write x — X) i>o ^Sj' where 5*0 is the set of 
indices of the s largest components, in absolute value, of x. Si 
is the set of indices of the s largest components, in absolute 
value, of X in 5*0, and so on. From the construction of the 
sets Sj,j > 0, we obtain that xg^ — Xg, ||x5<:||g — Cs g(x), 

E,>2l|x5,||2<.l/^-l/%s,g(x), 

||X5,||2 < .si/^-i/^||xs,_J|^^/^||x5j|f 

< SV2-V^||X5,_J 

for all J > 1, and 

|A(xso +xsj|| 



(A.ll) 



|Ax||^ = 



J>2 



I Ax 



Sj\\2 



-2 Y, ( Axso , Ax5, ) + 2 ^ ( Ax5, , Axs, ) 

J>2 j>2 



+2 



Yl (Axs^,Ax5^, 

■2<3<3' 



(A.12) 



RecalUng that | (Axs^ , Axs^, ) | < (52s(A)||xsJ|2||xs^, II2 for 
all j' y^ j (fl I), and applying the restricted isometry property 
(II. lib , we obtain from dA.Ub and (IA.12b that 

(l-<52s(A))||x||2 < ||Ax||2 + ^2,(A),si-2/9(as,g(x))2 

+2y2<52s(A),si/2-i/«||x||2as,g(x) 

< ||Ax||2 + <52s(A)e||x||2 

+<52s(A)(l + 2e-i)si-2/?(as,g(x))2, 



where e > 0. Then (IIII.ll i follows by taking e 

V2(<52.(A))-i+2. 
Similarly we get 



||Ax||2 < (l + ,52s(A) + v/2MA))l|x|| 



+ (1 + y2MAy)<52s(A)si-2/?(^^_^(x))2. 
This proves ( IIII.2l i. 

E. Proof of Theorem 13.21 

Take an s-sparse vector x £ M". Then x = x^ and 
(Ts.g(x) — 0. This together with the sparse approxima- 
tion property ^^ gives ||x||9 = Hx^H* < D\\A:>i.\\l + 
/3s«/P-V^,,(x)« = i^llAxll*, and hence proves (IIinT i. 

Take a 2s-sparse vector x € R" and write x — xs^, + 
X5j for some subsets 5*0 and Si of {l,...,n} with empty 
intersection and cardinality less than or equal to s. Applying 
(II.2I 1 to the given 2s-sparse vector x and replacing S by Sq 
and 5*1 respectively, we obtain 

||X5J|« < 7^||Ax||«+/3s«/^-i||xsJ|^ < D\\A^\\l + p\\^sA\l 

(A.13) 
and 

IIX5JI? < Z?||Ax||^+/3s^/P-i||x5j|^ < i?||Ax||^+/3||xso||^ 

(A. 14) 
Summing up the above estimates (|A.13l l and (|A.14| i yields the 
following inequaUty: 

(1-/3)11x11^ = (l-/3)(|lx5j|; + 11x5,11;)'/'^ 

< (l-/3)(|lx5jl« + 11x5,11?) <2i?|lAx|l^. 

Hence ( IIII.4l i follows. 
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